Finite-State Spell-Checking with Weighted Language and Error Models—Building and Evaluating Spell-Checkers with Wikipedia as Corpus
نویسندگان
چکیده
In this paper we present simple methods for construction and evaluation of finite-state spell-checking tools using an existing finite-state lexical automaton, freely available finite-state tools and Internet corpora acquired from projects such as Wikipedia. As an example, we use a freely available open-source implementation of Finnish morphology, made with traditional finite-state morphology tools, and demonstrate rapid building of Northern Sámi and English spell checkers from tools and resources available from the Internet.
منابع مشابه
State-of-the-Art in Weighted Finite-State Spell-Checking
The following claims can bemade about finite-statemethods for spell-checking: 1) Finite-state language models provide support for morphologically complex languages that word lists, affix stripping and similar approaches do not provide; 2) Weighted finite-state models have expressive power equal to other, state-of-the-art string algorithms used by contemporary spell-checkers; and 3) Finite-state...
متن کاملCreating and Weighting Hunspell Dictionaries as Finite-State Automata
There are numerous formats for writing spell-checkers for open-source systems and there are many lexical descriptions for natural languages written in these formats. In this paper, we demonstrate a method for converting Hunspell and related spell-checking lexicons into finite-state automata. We also present a simple way to apply unigram corpus training in order to improve the spellchecking sugg...
متن کاملIntegrating Dictionary and Web N-grams for Chinese Spell Checking
Chinese spell checking is an important component of many NLP applications, including word processors, search engines, and automatic essay rating. Nevertheless, compared to spell checkers for alphabetical languages (e.g., English or French), Chinese spell checkers are more difficult to develop because there are no word boundaries in the Chinese writing system and errors may be caused by various ...
متن کاملUsing Google to Create a More Accurate and Easily-Extensible Spell Corrector
Spell checkers are now a common, integrated part of many commercial and freely available word processing programs. Agglutinative languages (such as Hungarian and Finnish) pose a separate problem, as there are many different " correct " forms for any given word. Due to the seemingly infinite number of possible words, the limited scope of a dictionary (provided with most spell-checking software) ...
متن کاملBuilding a learner corpus
The paper describes a corpus of texts produced by non-native speakers of Czech. We discuss its annotation scheme, consisting of three interlinked levels to cope with a wide range of error types present in the input. Each level corrects different types of errors; links between the levels allow capturing errors in word order and complex discontinuous expressions. Errors are not only corrected, bu...
متن کامل